\section{Conclusions and future work}

We have described a modern implementation of the \commonlisp{}
\texttt{loop} macro.  The main benefit of our method is better
\emph{modularity} compared to existing implementations, which makes
maintenance easier, and also allows for more modular integration of
client-defined extensions.

Our implementation contains significantly more code than, for
instance, \mitloop{}; more than $5000$ lines compared to $2000$.  There
are several explanations for this discrepancy:

\begin{itemize}
\item Our code has more lines of comments; nearly $1500$ compared to
  less than $200$ for \mitloop{}.
\item Our implementation contains more than $300$ code lines
  dedicated to specific conditions and to condition reporters for
  those conditions, whereas \mitloop{} uses condition signaling very
  rarely, and then mostly using simple conditions with condition
  reporters in the form of literal strings.
\item Our implementation is divided into nearly $50$ files, or
  \emph{modules}, and each new file represents some overhead in terms
  of code size.
\item Our implementation contains more semantic verification as shown
  by the fact that it rejects the examples of non-conforming code shown
  in \refSec{sec-mit-loop}.
\item Commonalities between clause types are captured as explicit
  class definitions which require additional code.
\item We most likely have not identified all the instances where
  refactoring the code would be beneficial.
\end{itemize}

We believe that some ordinary code factoring will bring the difference
in code size (not counting comments) down to a small difference that
can be attributed to per-module overhead, and to the fact that we have
more extensive semantic verification.

\subsection{Use external parser framework}
\label{sec-using-external-parser-framework}

When we started the work on this library, we were unaware of any
existing libraries for combinator parsing written in \commonlisp{}.
Since then, we have been made aware of several libraries with such
functionality, in particular:

\begin{itemize}
\item ``cl-parser-combinators''%
\footnote{https://github.com/Ramarren/cl-parser-combinators} which is
a library for combinator parsing inspired by Parsec
\cite{Leijen:Meijer:Parsec}.  Parsec was originally written in
Haskell, and later re-implemented in other languages as well.
\item ``SMUG''%
\footnote{https://github.com/drewc/smug} which seems to be more
self contained than cl-parser-combinators, especially when it
comes to the documentation.
\end{itemize}

We plan to evaluate cl-parser-combinators and SMUG to determine
whether they provide the functionality required for parsing
\texttt{loop} clauses, and if not, whether any of them can extended to
obtain this functionality.

A significant advantage of using one of these libraries over the
existing technique is that they both have full support for the most
general backtracking capabilities of combinator parsing.  Using one of
them rather than our current technique would make it unnecessary to
consider careful ordering of clause parsers the way we currently need
to do.

A possible disadvantage might be that full backtracking is potentially
costly in terms of performance.  However, we do not expect performance
of clause parsers to be a determining factor for the overall
performance of a \commonlisp{} compiler.

\subsection{Second clause parser}
\label{sec-second-clause-parser}

As mentioned in \refSec{sec-our-technique}, we are able to signal
appropriate conditions in some cases when the initial attempt is made
to parse the body of the \texttt{loop} form as individual clauses.
However, when a syntax error is detected in some clause, all further
analysis is abandoned.  It would clearly be better if the analysis
could continue with the remaining clauses, and if an appropriate error
condition could be signaled for the faulty clause.

A simple way of improving error reporting would be to add more parsers
for each clause type.  These additional parsers would recognize
incorrect clause syntax and ultimately result in an error being
signaled, but more importantly, they would succeed so that parsing
could continue with subsequent clauses.  For example, a common mistake
is to omit the \texttt{=} character in a \texttt{with} clause
\seeapp{loop-syntax}.  A second parser would recognize this defective
\texttt{with} clause as valid, but then either signal an appropriate
error or issue a warning.

Unfortunately, however, while the parsing technique we use has many
advantages as described in \refSec{sec-benefits}, it also has the main
disadvantage that parsing gets slower as more parsers need to be
tried, in particular if no care is taken to order the parsers with
respect to probability of success.

We plan to avoid this conundrum by implementing a \emph{second parser}
for parsing individual clauses.  This second parser would be invoked
only when the first one fails.  In that situation, we estimate that
performance is of secondary importance and that emphasis should be on
appropriate error signaling.  Only this second parser would contain
the additional clause parsers that recognize common incorrect clause
syntax, and each such parser would be associated with an error or a
warning appropriate for the situation.  Existing clause parsers that
recognize correct clause syntax would be re-used, and additional
parsers for each clause type would be added to the set of parsers for
that clause type.

At the moment, we have no quantifiable estimate of the cost of a fully
backtracking parser, as compared to our simplified technique.  At this
point, we are also unable to quantify the additional performance
penalty of having additional parsers for incorrect clause syntax when
only correct \texttt{loop} syntax is parsed.  Further work is required
for a precise estimate.

\subsection{Alternative parsing techniques}

While a number of different parsing techniques such as LALR
\cite{DeRemer:lalr} or Pratt \cite{Pratt:1973:TDO:512927.512931} would
be feasible for the parsing the \texttt{loop} grammar as specified in
the standard \seeapp{loop-syntax}, an explicit goal of our work is to
make it possible for client code to extend the grammar in a modular
way.  Most parsing techniques require the full grammar to be available
a priori which is incompatible with this goal.  There are parsing
techniques other than combinator parsing that could be used, and we
plan to investigate the feasibility of such techniques, in particular
Earley \cite{Earley:1970:ECP:362007.362035} parsing.

\subsection{Code refactoring}

As suggested in the beginning of this section, there are very likely
several remaining opportunities for code refactoring.  Part of the
plan for future work is to identify such opportunities and restructure
the code accordingly, while respecting the existing modular structure
of the code.
